Skip to content

[Improve][Connector-V2] Add branch option for paimon sink #9982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

[Improve][Connector-V2] Add branch option for paimon sink #9982

wants to merge 6 commits into from

Conversation

@xiaochen-zhou
Copy link
Contributor

@xiaochen-zhou xiaochen-zhou commented Oct 25, 2025 •
edited
Loading

Purpose of this pull request

Add branch option for paimon sink


sink {
  Paimon {
    warehouse = "/tmp/seatunnel_mnt/paimon"
    database = "default"
    table = "st_test_p"
    branch = "test-branch"
  }
}

Does this PR introduce any user-facing change?

Yes

How was this patch tested?

Add new e2e tests: PaimonSinkBranchIT

Check list

@xiaochen-zhou xiaochen-zhou force-pushed the feature/writer-special-branch branch from e39f272 to 75f1d36 Compare October 26, 2025 02:23
@xiaochen-zhou
[Improve][Connector-V2] Add branch field option for paimon sink
11d2326 
[Improve][Connector-V2] Add branch field option for paimon sink

fix failures tests

[Improve][Connector-V2] Add branch option for paimon sink

[Improve][Connector-V2] Add branch option for paimon sink

fix e2e failures
@xiaochen-zhou xiaochen-zhou force-pushed the feature/writer-special-branch branch from 2d34841 to 11d2326 Compare October 26, 2025 08:11
@davidzollo
Copy link
Contributor

davidzollo commented Oct 26, 2025

Should PaimonSaveModeHandler also handle it?
Considering that schema_save_mode = "RECREATE_SCHEMA"

    Table paimonTable = ((PaimonCatalog) catalog).getPaimonTable(tablePath);

@xiaochen-zhou xiaochen-zhou force-pushed the feature/writer-special-branch branch from 9944ed5 to d0012dd Compare November 10, 2025 06:55
corgy-w
corgy-w reviewed Nov 10, 2025
View reviewed changes
...r-paimon/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/sink/PaimonSink.java Outdated
Comment on lines 114 to 116
if (!branchManager.branchExists(branchName)) {
throw new UnsupportedOperationException(
"Branch: " + branchName + " not exists");
Copy link
Contributor

@corgy-w corgy-w Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PaimonConnectorErrorCode
Error messages can be extracted

Copy link
Contributor Author

@xiaochen-zhou xiaochen-zhou Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PaimonConnectorErrorCode Error messages can be extracted

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@corgy-w corgy-w corgy-w left review comments

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

connectors-v2 document e2e paimon

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@xiaochen-zhou @davidzollo @corgy-w